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SYSTEM FOR GUIDING AlVffiDICAL INSTRIJMENT IN A PATIENT BODY 

FIELD OF THE INVENTION 

The present invetttion relates to a medical systenL The invention also relates to 

method to be used in said system. The mvention finds for example its application for guiding 
a catheter inside the heart of a patient during an electrophysiology interventional procedure. 

BACKGROUND OF THE INVENTION 

Clinical appHcations in which a medical instrument has to be guided into the body of 
a patient are becoming widespread. Notably Ihe glowing interest in minimatinvasive 
methods for the treatment of cardiac diseases necessitates the development of methods and 
devices allowing the physician to guide a medical instrament to predetermined positions 
inside or outside the heart In electrophysiology for example, it is necessary to gmde a 
cathet^rtoapluralityofpredeterminedpositionsontheventricularora^ walls inorderto 

measure an dectrical pulse or to bum wall tissues. 

US patent 6,587,709 discloses a system for guiding a medical instnunent m the body 
of a patient Such a systom acquires a live 3D ultrasound image data set using an ultrasound 
probe. An advantage of acquiring a 3D hnage data set is to get depth information. An 
- Lantage of usmg a live 3D ultrasound imaEP modality is that the surrounding anatomy IS 

visible which fecilitates the guidance of the medical instrument by the physician. TTie system 
furlher'comprises localization means for localizing the medical instrument wiflrin lie 3D 
vitxasound data set. which locates three ultrasound receivers mounted on the medtcal 

ir^ent relatively to said ultrasound probe. Such a localization allows ^ 
selection of a plane to be imaged, whidi comprises at least a section of the medical 
i^cr^. Therefore no readjustment of the ultrasound probe position by hand is necess^y. 
AfirstdrawbackofsuchaSDultrasounddatasetistohaveanarrowviewmgfield, 

which does not cover the whole part of the patient body concerned by a catheter introduction 
andplacementTherefor^'fc^guidiigthecatheterduringthewholeprocedure^ 

ultrasound probe has to be moved several times. At each displacement, a preoperative step 

1 1^4^. mrvm ist needed, because 



of locating me uiu»«»u««i:'i«"v,»*-*~~'' . 

the location of fte catheter is measured relatively to the ultrasound probe location. Such a 
pre-operative step may delay and compUcate tixe interventional procedure. 
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A second drawback of the ultrasound imaging modality is to have a low resolution 
Theiefore, the acquired 3D ultrasound data set does not give an image of the catheter and its 
surrounding of acceptable quality. 

A third drawback of the ultrasound imaging modality is that there ate some zones of 
the patient body where the thoracic cage blocks the ultrasound scan and no ejq)loitable image 
canbeou^ut 

SIJMMARY OF THE INVENTION 

The object of the invention is therefore to provide a system for guiding a medical 
instrument in a patient body, which gives an improved visibiUty of the medical instrument 
and its surrounding anatomy during the whole procedure! 
This is achieved by a medical system comprising: 

- a medical instrument to be guided in a patient body, 

- X-Ray acquisition means for acquiring a two-dimensional X-ray image comprising a 
projection of said medical instrument in accordance with a geometry of said X-Ray 
acquisition means, 

- ultrasound acquisition means for acquiring a three-dimensional ultrasound data set of said 
medical instrument using an ultrasound probe, 

- means for localizing said ultrasound probe within a referential of the X-ray acquisition 
means, 

- means for providing a first ultrasound localization of said medical instrument within a 
referential of said ultrasound acquisitian means, 

- means for converting said &st ultrasound localization within said referential of the 
ultrasound acquisition means into a first X-ray localization within said referential of the 
X-ray acquisition means, using said localization of the ultrasound probe, 

- means for providing a second X-ray localization of said projection of the medical 
instrument in a referential of said two-dimensional X-ray image, 

- means for mapping said three-dimensional ultrasound data set with said two-dimensional 
X-ray image in accordance with a transformation, which n^iindmizes a distance between a 
projection of said first X-my localization on said two-dimensional X-Ray image in 
accordance with said geometry of the X-Ray acquisition means and said second X-ray 
localization. 
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meam for generating and dispkying a bi-modalrepRi5entation of said medico 
instrument in whidi said two-dimensional Krxsy image and said mapped three-dimensional 
ultrasound data set ate combined. 

With the inveirtion, a bimodal representation is provided, in which two-dimensional 
(2D) X-Ray data and thiee-dimtaisional (3D) ultrasound data are combined. 2D X-ray data 
provide a good visibiUty and a high resolution of the medical instrument and of bone 
structures. 2D X-Ray data also benefit ftom a large viewing field, which allows a 
visualization of the whole area of tte patient body concerned by the electiophysiology 
procedure. 

3D ultrasound data provide a good visibility of soft tissues and vascularities m a 
surrounding of the medical instrument. In addition, 3D ultrasound data give an indication of 
depth, which is Tct provided by the 2D X-Ray image, because said XRay image only 
provides a projection of said medicai mslrument in accordance with a geometry of the X-Ray 
acquisition m«ms. Sudi a geometry defines lines of projection, along which absorptions of 

X-rays by the exposed tissues of the patient are accumulated. 

Therefore, the visimty of the surrounding of the medical instrument is irnproved by 

the combination of die 2D X-Ray and the 3D ultrasound data. 

In order to provide such a combmation. the system firstly localizes the ultrasound 
probe and tiie 3D ultrasound data set in a referential of the X-Ray acquisition means. Such a 
referential of tiie X-ray acquisition means is supposed to be fixed. Therefore, assuming that 
the ultrasound probe does not move, a position of any pomt of die 3D ultrasound data set is 
known in said referential flie X-ray acquisition means. 

The system m accordance mth the invention fortiier provides a first ultrasound 
locaUzation of the medical instrument in tiie 3D ultrasound data set. Such a first ultrasound 
localization is expressed with coordinates of a referential of die 3D ultrasound acquisdtion 
means. The first ultrasound localization is tiien converted tato a first X-Ray locahzation of 
the medical instmment witiiin the referentid of the X-Ray acquisition system, using ti»e 
localization of the ultrasound probe. 
) The system in accordance with the invention also provides a second X-Ray 

localization of the projection of flie medical instrument in die 2D X-ray image, which is 
caressed wifli coordmates of a referential of die 2D X-Ray image, for mstance a referential 
of tive detector. Such a referential is known by the geometry of die X-Ray acquistion means. 
Therefore, die geometry allows to detetmine a projection of any point of the referential of die 
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X-Ray acquisition means and, inveaisely, a point of the detector corresponds to a line of 
projection within the referential of the X-Ray acquisition means. 

From said first X-Ray and second X-Ray localizations the mapping means are 
intended to define a transformation, which minimizes a distance between a projection of said 
S first X-ray localization on the two-dimensional X-Ray image in accotdance with the 
geometry of the X-Ray acquisition means and said second X-ray localization. Such a 
transformation is applied to the 3D ultrasound data set Finally the system generates and 
display a bimodal representation in which the 2D X-ray image and the transfonned 3D 
ultrasoimd data set are combined by affecting to a point of the bimodal representation either 

10 ultrasound data or X-ray data or a combination of both. 

An advantage of such a transformation is to compensate for errors in the localization 
of the ultrasound probe. These errors may be due either to any external or internal movement, 
like respiratory movements, which could have occurred after the localization of the 
ultrasoimd probe in the referential of the X-Ray acquisition system, or to an imprecision in 

1 3 the localization of the ultrasound probe, for instance related to its orientation. Conseque ntly 
the mapping of 3D ultrasound and 2D X-ray data in the surrounding of the medical 
instrument is made more precise. In particular, the distance between the medical instrument 
and the wall tissues shown by the bimodal representation becomes more accumte and 
reliable, which is of high interest for guiding the medical instrument into contact with a wall 

20 tissue. 

' In a first embodiment of the invention, &e localization of the medical instrument in 

■ 

both 3D ultrasound data set and 2D X-Ray image is based on a detection of one landmark, for 
instance a tip usually placed at one extremity of the medical instrument Such a localization 
allows to define a translation for mapping the 3D uhrasound data set with the 2D X-ray 
25 image. An advantage of this first embodiment is that it is very simple and easy to implement 

In an alternative, the system in accordance with the invention further con^rises 
means for detecting an orientation of the medical instrument, which is defined by two Euler 
angles. Therefore, a transformation can be specified, which comprises a translation and two 
rotations. 

30 In a second embodimmt of the inv^ition, the first and second localizations of the 

medical instrument are based on a plurality of landmarks, which are for instance arranged at 
different places on the medical instrument. An advantage is that a transformation comprising 
a translation and 3 rotations can be defined, which is enough to completely specify a 
displacement of the 3D ultrasound data set in the referential of the X-Ray acquisition means. 
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Therefore, in a sunounding of the medical instniment, the mapping of ultrasound and X.iay 
data is improved. 

In a third embodiment of the mvention. the plurality of landmarks are placed on the 
medical instrument and on at least two reference medical instruments. An advantage is that 
5 the two reference medical instrmnents are expected to be fixed Consequentiy, any 

displacement of a landmark of a reference medical mstrument with respect to the anal«my 
may advantageously be considered as an indication that the ultrasound probe has moved and 
more generally that the mapping of the 3D ultrasound data set with the 2D Xray image is no 
more reliable. Another advantage is that the landmarks used forlocalizing the medical 

10 instnmient are more distant W each other. Iherefoxe, the definition of the tt^ 

morerobusttolccalerrorsof localization. Therefore, a mapping transformation can be 
defined, which appUes to a larger surrounding of the medical instrument and the precision of 
the bimodal representation is unproved on a larger area. , , , 

These and other aspects of the mvention wiU be apparent firom and will be elucidated 
with reference to the embodiments described hereinafler. 



15 



20 



25 



30 



BRIEF DESCRIPTION OF THE DRAWINGS 

The present invention will now be described in more detaU, by way of example, with 
reference to Aeaccompanymg drawings, wherem: 

. Fig lis a schematical drawing of a system m accordance with the mvention. 

Fig" 2 is a schematical drawing of means for localizing the ultrasound probe wi^^^^ 

ray referential, when the ultrasound probe is equipped with acdvel«^ 

fL 3 4a and 4b are schematical drawmgs of means for localizing the ultrasound probe 

and 'the 3D ultrasound data set within the referential of the XRay acquisition m^ 
whentheultrasoundprobeisequippedwithabehcomprisingradio-opaqu^^ 

. Fig,5isasd.emadcaldcawingofmeansforprovidmgafirstloca3izationofthem 
instroment within the 3D ultrasound data set. 

Fig 6 is a schematical drawing of means for providmg a second locahzation of the 
projection ofthe medical instrument in a referential ofthe 2D X-tay image, 

projecuon ui ui ^„„o fcr mauomB the 3D ultrasound data set 

Fig 7 is a schematical drawing of mapping means for mappmg 
with the 2D X-ray image when the transfomiation is a translation. 
. Hg8isaschematicaldrawingofmeansforprovidingafirstlocaliza.onofthem^^ 

inLnent within the 3D ultrasomid data set, when a pluraUty of landmarks are placed on 
the medical instrument and two reference instruments. 
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- Fig. 9 is a schematical dxawing of means for generating a bimodal representation in 
accordance with the iavention, 

" Rg- 1 0 is a schematical drawing of means for generating a bimodal representation, when 
the system in accordance with the invention comprises means for segmenting a wall 
5 tissue region around the medical instrument, 

- Fig. 11 is a functional diagram of the method in accordance with the invention. 

DETAILED DESCRIPHON OF THE INVENTION 

1 0 The present invention relates to a medical system conq)rising a medical instrument to 

be guided in a patient body and data acquisition and processing means for visualizing said 
medical instrument. Such a system is particularly adapted for guidiag a catheter within the 
heart cavities in order to diagnose and cure heart diseases, but it can more generally be used 
for guiding any other medical instrument in the patient body, like for instance a needle. 

15 The schematical drawing of Fig. 1 shows a patient 1, who is arranged on a patient 

table 2 and whose symbolically indicated heart 3 is subjected to a treatment by means of a 
catheter 4 introduced into the body. The system con^xises means 5 for acquiring a 2D X*ray 
image of the patient body. Said X-ray acquisition means comprise a focal X-ray source 6 and 
a detector 7. Advantageously, these X-ray acquisition means 5 comprise a C- arm system, as 

20 it is usually the case in a cathlab room. An advantage of such a C-arm system is to be able 
to have a rotational motion around the patient body in order to produce a plurality of 2D X- 
ray images of the patient at known orientation angles. 

The system in accordance with the invention furth^ comprises means 8 for acquiring 
a 3D ultrasound data set from an ultrasound probe 9, which has been placed on the patient 

25 body and jSxed by fixation means, for instarice a belt 10 or a stereotactic arm. It should be 
noted that both 2D X-ray image and 3D ultrasound data set are acquired in xeattime, which 
enables a Uve visualization of the medical instrument, when it is guided inside the paAeat 
body. 

The X-ray acquisition means 5 comprise a referential of coordinates (O, x, y, z), 
30 called X-Ray referential hereinafter, in which the geometry of the focal X-ray source 6 and 
the detector 7 is knowiL It should be noted that the X-Ray referential (O, x, y, z) is bound to 
the fixed part of the X-Ray acquisition means and not to the C-arm. Therefore, the orientation 
of the C-arm can be expressed in said X-Ray referential. However, the geometry of the X- 
Ray acquisition means is dependent on a particular position of the C-aim. 
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The system in accordance with the invention further ctmqMises means 11 for 
localizing the ultrasound probe 9 within the X-Ray referential (O, x. y, z), means 12 for 
providing a first ultrasound localization Loc.us of the catheter 4 in the 3D ultrasound data set 
within a referential of the ultrasound acquisition means, means 13 for providing a second X- 

5 Ray localization Loc^, xr of a projection of the catheter 4 m the 2D Xray image within a 
referential of the 2D X-Ray image or of the detector, means 14 for converting said first 
ultrasound localization Loci.us into a first X-Ray localization of the medical instrument 4 
within the X-Ray refraential. means 1 5 for mapping said 3D ultrasound data set with said 2D 
X-r^ image m accordance with a transformation, which minimizes a distance between a 

10 projectionofsaidfirstX-Raylocaiizationonthe2DX-Rayimageinaocordancewiththe 

geometry of the X-Ray acquisition means and said second X-Ray localization. The system in 
accordance with fee invention finally comprises means 16 for generating and displaying a bi- 
modal representation BI of the catheter 4 in which the 2D Xray hnage and the mapped 3D 
ultrasound data are combined. TTie bhnodal image BI is displayed on a screen 17. 

1 5 Referring to Fig. 2, the piobe localization means 1 1 ate, in a first approach, based on 

an active localizer 1 5, welWmown to those skilled in the art. which is arranged on the 
„ltn.soundprobe9. Said active locaHzer 18, for instance an RF coU. is intended to transmit 
an RF signal to an RF receiving unit 1 9 placed under the patient body and for mstance 
urtegrated into the table, m RF receiving unit transmits the r^ 

20 means20formeas«ringapositionoftheultrasoondprobe9maknownreferential.for 

instance the X-Ray referential (O. x. y. z). It should be noted that the active localizer 18 must 
be two-dimensional and placed on fee ultrasound probe 9 m such a way that a precise 
measurement of the position and of the orientation of the ultrasound probe can be calculated. 
An advantage of this first approach is to provide a precise localization of the ultmsound 
25 probe 9. 

m a second approach of the probe localization means 1 1 shown in Fig. 3. the 
ultrasomid probe 9 is fixed around the body of die patient 1 with a belt 10 equipped with at 
least fliree non aligned interdependent radio-opaque markers M,. M2 andM,. For mstance. 
the belt 10 comprises a plexiglas plaque 21, in which the three non aligned interdependent 

30 radio-opaque markeis are fixed. 

^ ThethreemarkersM,,M2andM3belongtoa8ameplane,thereforeatleasttwo2D 

different X-ray projections 2DXR, and 2DXR2 acquired with different orientation angles 6. 
and 02 of the C-ami system 5 are needed in order to determine the position of the ultrasound 
pxobe in the X-Ray referential (O. x. y, z). However, since the three markers are 
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interdependent, and non-aligned, which means that they fonn a rigid tetraedre, it is well- 
known to those skilled in the art that the position of the probe is completely specified by the 
two different X-xay projections 2DXRi and 2DXR2. 

Referring to Figs. 4a and 4b, we consider the detector referential (dO, dx, dy). It will 
appear to those skilled in Ihe art that six piaiameters like for instance the coordinates (dxi , 
dyi), (d3a, dya), (dxs, dj^) of the projectians Pi, P2, P3 of the three markers Mi, M2 and M3 in 
the first 2D X-ray image 2DXRi and the coordinates (d'xi, d'yO> (A"^> d'y2),.(d'x3, d'ya) of 
the projections P* 1 , P'2, P'3 of the three marireis Mi , M2 and M3 in the second 2D X-ray 
image 2DXR2 do completely specify flie position of the ultiasound probe 9 in flie X-Ray 
referential (O, x, y, z), provided that the difference of orientation angle between the two X-^ 
ray projections is known . Moreover, it should be noted that the localized points Pi, P2 P3 
and P'l, P'2 and P'3 follow epipolar constraints: this means for instance that a line Lj linking 
the source focal point to Ae point Pi appears as ia projected hne L'j m the second X-ray 
image 2DXFfe, ^Nido\% comprises P'l . A fibcst advantage is that P'l has not to be searched 
within the whole hnage, but only on the projected Ime L*i . A second advantage is that it 
gives a way of assodatmg the points Pi, P2, P3 and P'l, P'2, P'3 with the right markers Mi , M2 
and M3, 

An advantage of the radio-opaque markers Mi, M2 andMs is to appear in a 2D X- 
projection with a very high contrast, which makes their localization easy and precise. Such a 
localization may be achieved manually or automatically. In the manual case, a user may dick 
on at least two radio-opaque markers in each 2D X-ray projections. In the automatical case, 
image processing techniques well known to those skilled in the art, like for mstance a 
morphological filter, may be used for detecting the radio-opaque markers, which appear as 
highly contrasted blobs in the 2D X-ray projections. 

It should be noted that such a localization of the ultrasound probe 9 is firstly handled 
m a preoperative step of a clinical procedure. As a matter of feet, with the mvention, there is 
a priori no need to move flie ultrasound probe 9 during the clinical procedure, because the 
large field of view of the X-ray acquisition system allows a visualization of the whole part of 
the patient body concerned by the clinical procedure. However, unwanted motion of the * 
probe may occur due to a patient movement Therefoie, in order to avoid any error 
accumulation, the probe localizatim has to be repeated regularly during the clinical 
procedure. 

Once the ultrasound probe 9 has been boated m the X-Ray referential (O, x, y, z), an 
orientation of the probe is known and therefore, the location of the 3D ultiasound data set 22, 
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also caUed 3D ultrasound cone, can be deduced. This is achieved byHie converting means 
which calculate a position of a point of said 3D ultaasound data set in the X-Ray referential, 
from said ultrasound probe localization. A projection of said point on the detector can also be 

deduced. 

Referring to Fig. 5, the first localization means 12 are intended to provide a first 
localization Loc, us of the medical instrument in the 3D ultrasound data set within the 
referential of the ultrasound acquisition means (O'. x% y'. z'). The detection means aEow to 
automatically define a crop plane 30 by the detected point T and a normal orientation N . 
which corresponds to theknown orientation 32 of the X-ray source 6. An advantage is that, m 
view of generating a bimodal representation of the medical mstmment. the crop plane 30 can 
be used for delimitating a subvolume of interest within the 3D ultrasound data set and for 
removing all other data, which could occlude structures of interest like the medical 
instrument 4. This predefined crop plane 30 may also be advantageously rotated for 
searching a viewing angle view within the 3D ultrasound data set from which the medical 
instrument is more visible. A rotated crop plane is obtained. Advantageously said viewmg 
angle is appUed to the Caim system in order to optimize the 2D X-iay image. 

Referring to Fig. 6. the second localization means 13 ate intended to provide a second 
localization Loo^ of the projection of the medical instrument in the 2D X-ray imag. within 
the detector referential (dO, dx. dy) in accordance with the X-Ray geometry. 

Refcning to Fig. 7, the first ultrasound localization Loc,.us withm said referential of 
fl»e ultrasound acquisition means is converted into a first X-ray localizati™, Loc,^ within the 
X-Ray referential by the converting means 14. 

The localizations Loc.^ and Loo, XR ai« fi«ther used by the mapping means 15 for 
defining a transfomutionTr. which maps the 3D uhrasound data set with said 2D X-ray 

image A mapped 3D ultrasound data set is obtained. Such a transformation is defined such 
that a distance between the projection of the first X-Ray localization on the 2D X-ray irnage 
in accordance with the X-Ray geometry and the second X-Ray localization is minimized.. 

It should be noted that the first and second X-Ray localizations Loc,. xR, Looz^R rx>^ 
) c«n^risescveralfeat«resHkeapositionofalandmarkin1heXRayrefere«tial.an 

orientation of the medical instrument or any other characteristic of the shape of the medrcal 
instrument 4 . Consequentiy . the way in which such a distance is measured may depend on the 
features used for defining the first and second localizations. With a single landmark, an 
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EucUdean distance may be sufficient With a plurality of landmarks, functions of distance, 
which are well known to those skilled in the art, may advantageously be used. 

It should be also noted that these first and second localizations Loci, xr, Loc2^ of the 
medical instrument are obtained in realtime and continuously during the clinical procedure, 
5 thus allowing a reattime mapping of the 3D ultrasound data set with the 2D X«ray image, 
whidi is based on a tracking of the medical instrument 4. 

The medical instrument usually comprises a tip T at its extrraiity. In particular, an 
electrophysiology catheter comprises a metal tip, which is very echogen and leaves a spedJBc 
signature in the 3D ultrasound data set. Such a metal tip is also strongly radio-opaque. 

1 0 Therefore, such a metal tip presents a high contrast both within the 3D ultrasound data set 
and the 2D X-ray image and can be advantageously considered as a valuable lanHmg^rir in 
addition, the tip of a catheter is a small and thin segment Therefore, either the tip end is 
considered as a punctual landmark or the whole tip is considered in order to specify at least a 
punctual landnmk and an orientation of the medical instrumeixt. 

15 - Therefor^ the detection means in accordance with the invention involve image 

processing techniques^ which are well known to those skilled m the art, for enhancing either a 
highly contrasted punctual blob or a highly contrasted segment in a relative^ uniform 
backgrovmd. 

In a first embodiment of the invention illustrated by Figs. 5 and 6, the localization 
20 means 12, 13 comprise means for detecting the tip end of the medical instrument 4. In the 
following the tip end will be denoted T within the 3D ultrasound data set and the tip 
projection will be denoted Tp in the 2D X-ray image. The Hp end T is detected at a position 
(xiT, yiT, 2it) in the X-Ray referential and the projection T? of the tip is detected at a position 
(dxT, dyT) in the detector referential (dO,dx,dy). In the first embodiment of the uivention, 
25 the first and second localizations Locj^ xr> Loc2;xr are based on the respective positions of the 





m 


iTi 





Consequently, fi-om the knowledge of these first and second localizations Loci, xk, 
Loc2^R, the mapping means 15 in accordance with the first embodiment of the invention are 
capable of defining a translation for minimizing the distance D between a projection P(Loci, 
30 xr) of the first X-Ray localization Loci, xr and the second X-Ray localization Loc2,xr> as 
shown in Fig. 7. Such a projection P(Loci, xiO> which is defined by the geometry of the X- 
Ray acquisition means, belongs to a projection line 37 passing through the tip end T. An 
advantage of this first embodiment of the invention is that it is very simple. 
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The translation defined by the transfonnation means is specified by a vector Tr , 
which connects the tip T to the projection line 36. It turns out that a plurality of translations 
can be derived from such a definition. Preferably, the chosen translation is tiie one which 
niinimizes a 3D displacement of the first X-Ray localization Loci. xr. This particular 
5 translation is defined by the vector fr . which is perpendicular to the projection line 36. 

It should be noted that, due to the conic geometry of tiie X-Ray acquisition system, 

the vector fr is not necessarily inchided in the crop plane 30. 

In an allemative of the first embodiment of the invention, flie whole tip is detected, 
which allows to determine a location of a landmark, for instance the tip end T and an 
10 orientation of tixe tip, specified by two Euler angles. Advantageously, a transfonnation 
comprising a translation and two rotations can be derived and the mapping of tiie 3D 
ultrasound data set with the 2D X-Ray image is inqnoved. 



15 



In a second embodiment of the invention also illustrated by Fig. 5, the first and 
second localizations of tiie medical instrument 4 are based on the detection of aplurality, i. e. 
at least tiiree non aUgned lantoks T. IJC2. IJs3, which are ananged on tiie medical 

mstrument 4. Such a ptorahty of landmarks aUow to define a second crop plane 33 and a 
second nonnal within the 3D ultrasound data set, which can advantageously serve for 
reorienting ti>e X-ray source 6 in order to optimize tiie X-ray acquisition witii respect to tiie 
20 detected position of tiie medical instrument 4. An advantage of tiie second embodiment of ti« 
invention is that it aUows to define a transformation havmg ax degrees of fi^edom. i. e. a 
translation and tiiree angles. Such a transformation completely specifies tiie displacement of 
the 3D ultrasound data set in tiie X-Ray referential. Therefore, tiie mappmg of the 3D 
ultrasound data set wifli tiie 2D X-Ray image is made more precise. 



25 



In a fliird embodiment of tiie invention illustrated by Fig. 8, the phirality of landmarks 
are distributed over tiie medical instrument 4 and at least two reference medical instruments 
40, 41. Said reference medical instruments 40. 41 are both fixed in the patient body during 
thi whole clinical procedure and comprise each an echogen and radioopaque tip T2, T3. They 
30 may also comprise other landmaAs ti«n tiie tips T. T2, T3, which may allow for instance tiie 
determination of tip orientations 0„ <?i , O3 . 

A first advantage of tiie tiiird embodiment of the invention is tiiat flie hindmarks used 
for localizing the medical instrument are more distant fix>m each otiier. Therefore, tiie 
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definition of the transfonnation is more robust to local errors of localization. As a matter of 
fact an eiror of one or two pixels has no consequences at a vicinity of the medical instrument, 
but canhave dramatical eJOTects in mote distant areas of the 3D ultrasound data set 

A second advantage of using landmarks which are located on the reference medical 
5 instruments is lliat, unlike the medical instrument 4, they are fixed with respect to the 

anatomy. Ccmsequenfly, any displacement of a landmark of a lefetence medical instrument 
with respect to the anatomy may advantageously consideied as an indication that the 
ultrasound probe has moved and more generally that the mapping of the 3D ultrasound data 
set with the 2D X-ray image is no more reliable and accurate. Notably, if one of Hie 

10 landmarks of a reference medical instrument is no more visible within the bimodal 

representation BI at a time t, a consequence is that the whole processus should be repeated, i 
e. a new localization of the ultrasoimd probe within the X-Ray referential should be 
performed. However, if none of the landxnadcs has disappeared at a time t, but was only 
di^laced with respect to its localization at a time tO, a motion compensation of the 3D 

15 ultrasound data set between times t and tO should be sufficient. 

It should be noted that for all the previously described embodiments of the invqation, 
the transformation is preferably chosen such that it minimizes a 3D displacement of the first 
X-Ray localization Loci^xr* An advantage is that such a transfbrxnation, which is intended to 
provide a small correction of a previous mapping of the 3D vdtrasoimd data set and the 2D X- 

20 Ray image, ensures that the landmarks of the first X-Ray localization will still be associated 
with tiie right landmarks of the second X-Ray localization of the medical instrument. 

The generation and display means 16 in accordance with the invention are intended to 
generate a bimodal representation BI of the medical instrument 4, in which information 
coming firom both the 2D X-ray image 2DXR and the transformed 3D ultrasound data set are 

25 combined. 

Preferably, such a combination is X-Ray driven, which means that it is made on the 
basis of a 2D X-Ray image 40, as shown in Fig. 9. 

Advantageously a 2D ultrasound view 41 corresponding to the ultrasound informaticm 
contained in one of the previously defined crop planes 30, 33 including at least part of the 
30 medical instrument 4 is extracted fi:om the 3D ultrasound data set 21 acquired at a time t 

A conespondance between points included into the 2D ultrasound view 41 and points 
included into the 2D X-ray insage 40 can be calculated fixnn the knowledge of the 
localization of the ultrasoimd probe 9 within the X-Ray referential (O, x, y, z) provided by 
the probe locaUzation means 11. 
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The bimodal projection is for instance fonned such that the intensity values of all the 
points of the 2D X-ray projection 40 which have a conesponding point in the 2D ultrasound 
view 41 are replaced. An advantage is that the bimodal projection 45 obtained offers both an 
inq)roved visibility of the surrounding tissues. 

It is wdl known to those skilled in the art that the projection of the medical 
instrument given by the X-iay source 6 on the detector 7 is of good quality and benefits ficom 
high resohition and contrast A position of the projection of the medical instrument 4 witiiin 
the 2D X-iay projection 40, that is in the detector referential (dO, dx, dy), can be derived 
from the position of tiie medical instrument in the X-Ray referential (O. x, y. z) given by the 
localization of the medical instrument within tiie 3D uhrasound data set by the ultrasound 
localization means 12, the. This position is for instance a set 43 of points of the X-ray 
projection corresponding to a set of points 42 within tiie 2D ultrasound view 41. 

Advantageously, tiie mtensity vataes of tiie pomts of tiie 2D X-ray projection 40 
belonging to tiie detected medical instrument are not replaced by tiie conespondmg 
uhrasound mtensity values. An advantage is to keep tiie good visibiUty and resolution of flie 
medical mstrianent provided by tiie X-ray acquisition means. 

In an alternative shown in Fig. 10. tiie system in accordance wifli tiie invention fiirflier 
conq)rises means for segmenting a wall tissue region, for instance tiie endocardiac waU 44 m 
tiie neigbourhood of flie medical instrument 4. This is achieved by image processing 
techniques, well known to tiiose skilled in flie art, such as mtensity value tiiiesholding. since 
waU tissues like myocardium appear brighter tiian blood m ultrasound images. 

Anoflier possilality is to use an active contour technique (also called "snake"). This 
technique, well known to tiiose skilled in tiie art. firstiy consists in defining an mitial contour 
and secondly m makmg said initial contour evolve under flie influence of internal and 
external forces. A final contour.46 is obtained. It is tiien possible to differentiate points 
located inside from pomts located outside flie contour 46 and to replace only tiie outside 
points of tiie 2D X-ray projection 40 by flie conespondmg points of tiie 2D ulti:asound view 
41 . An advantage of tins second embodiment is to benefit fiom X-ray information in a larger 

neighbourhood of the medical instrument 4. 

In anotiier alternative of tiie invention, an alpha blending technique, well- 
known to tiiose skilled in tiie art. is used for combming flie X-ray intensity vahies of tiie 
points of file X-ray projection witii tiie ultrasound intensity vahies of tiie corresponding 
pomts of tiie 3D ultrasound data set An advantage is tiiat fliis alternative is very shnple to 
implemeixt. 
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It should be noted that the generation means 16 could inversely generate a 
bimodal representation on fbe basis of the 3D ultrasound data set and replace X-ray 
information by ultrasound information. However, it is of less interest, because in this case, 
the bimodal representation has an image field which is reduced to the one of the 3D 
ultrasound acquisition means. 

It should be noted that the system in accordance with the invention presents a 
particular interest for electrophysiology procedures, which consist either in generating an 
electrical activation map of a heart cavity wall for diagnosing heart diseases or in burning a 
zone of the wall tissue, which has been identiJBed as abnormal. As a matter of fact, the system 
in accordance wifli the invention both provides a live visualisation of a large viewing field of 
the intervention, m which the medical instrument, the bone structures and the surrounding 
wall tissues are shnultaneously visible and a five localization of the medical instrument 
allowing a genemtion of the electrical activation m^ without furrier operation. 

The invention also relates to a method of guiding a medical instrument 4 in a 
patient body. Referring to Fig. 1 1, such a method comprises the stq)s of: 

- acquiring 60 at least a two-dimensional X-ray image, said two-dimensional X-ray hnage 
comprising a projection of said medical instrument in accordance wifli a geometry of said 
X-ray acquisition system, 

- acquiring 61 a three-dimensional ultrasound data set of said medical instrument 4 using 
said ultrasound probe 9, 

- localizing 62 said ultrasound probe m a wdferential (O, x, y, z) of said X-ray acquisition 
system, 

- providing 63 a first localization Loci, us of said medical instrument 4 withm a referential 
(0',x',y',z') of said 3D ultrasound acquisition means, 

■ converting 65 said first localization Loci, us within said refeiratial of the 3D ultrasound 
data set into a first converted localization Locj, xr within said inferential of flie X-ray 
acquisition system, 

providing 64 a second localization L0C2, xr of said projection of the medical instrument in 
said two-dimensional X-ray image within the referential (dO, dx, dy) of said 2D X-ray 
image, 

mapping 66 said three-dimensional ultrasound data set with said two-dimensional X-ray 
image in accordance with a transformation, which mininodzes a distance between a 
projection of said first X-Ray localization on said two-dimensional X-Ray image in 
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accordance with said geometry of the X-Ray acquisition means and said second 

4 

localization, 

. generating and displaying 67 a bimodal representation of said medical instrument 4 in 
which both 2D X-iay image and said m^ped 3D ultrasound data set are combined. 

drawings and their descriirtion hereinbefore iUustrate rather than limit the 
invention. It wiU be evident that there are numerous alternatives, which fall within the scope 
of the appended claims. In this respect the foUowing closing remarks are made: there are 
numerous ways of implementing fbnctions by means of items ofhaidwaie or software, or 
both. In this respect, the drawings are very diagrammatic, eadi representing only one possible 
embodiment of the invention. Thus, although a drawing shows different functions as different 
blocks, this by no means excludes that a smgle item of hardware or software carries out 
several functions, nor does it exclude that a single function is carried out by an assembly of 

items of hardware or software, or both. 

Any reference sign in a claim should not be constmed as limiting the claim. Use of 
the verb "to cwnpris^ and its conjugations does not exclude the presence of elements or 
steps other than those stated in a claim. Use of the article "a" or «an» preceding an element or 
step does not exchide the presence of a plurality of such elements or steps. 
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CLAIMS 

1 . A medical system comprising: 

- a medical instrument to be guided in a patient body, 

. X-Ray acquisition means for acquiring a two-dimensional X-my image comprising a 
5 projection of said medical instrument in accordance with a geometry of said X-Ray 
acquisition means> 

- ultrasound acquisition means for acquiring a three-dimensional ultrasound data set of said 
medical iiBtrument using an ultrasound probe, 

- means for localizing said ultrasound probe within a referential of the X-ray acquisition 

10 means, 

- means for providing a first ultrasound localization of said medical instrument within a 
referential of said ultrasound acquisition means, 

- means for converting said first ultrasound localization within said referential of the 
ultrasound acquisition means into a first X-ray localization within said referential of the 

15 X-ray acquisdtion means, using said localization of tie ultrasound probe, 

- means for providing a second X-ray localization of said projection of the medical 
instrument in a referential of said two-dimensional X-ray image, 

- means for mapping said three-dimensional ultrasound data set with said two-dim^ional 
X-ray image in accordance with a transformation, which minimizes a distance between a 

20 projection of said first X-ray localization on said two-dimensional XRay image in 

accordance with said geometry of the X-Ray acquisition means and said second X-ray 
localization^ 

. means for genemting and displaying a bi-modal representation of said medical instrument 
in which said two-dimensional X-ray image and said mapped three-dimensional 
25 ultrasovmd data set are combined. 

« * - 

2. A system as claimed in claim 1, wherein said means for providing a first ultrasound 
localization and said means for providing a second X-Kay localization of said medical 
instrument comprise detection means for detecting localization features of said 

30 medical instrument. 

3. A system as claimed in claim 2, wherein said localization features comprise a 
landmark of said medical instrument 
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4. A system as claimed in claim 3, wherein said tcansfonnation coniprises a translatioa 

* 

5. A system as claimed in claim 2, wh^ein said localization features comprise a 
pluiality of landmatfrs of said medical instrument 

6. A system as claimed in claim 5, wherein said transformation comprises a translation 
and three rotations. 

7- A system as claimed in claim 1 > wherein said transformation is intended to tninimiV^ 
a three-dimensional displacement of said first X-TAay localization. 

8. A system as claimed in claim 5, wherein said plurality of landmarks belongs to said 
medical instrument and to at least a first and a second reference medical instruments. 

9. A system as claimed in claim 1, wherein said ultrasowid piobe localization allows to 
define a crop plane, which delimitates in the 3D ultrasound data set data to be 
removed fiiom data to be used by the generating and display means for geneirating said 
bimodal representation, 

10. A method of guiding a medical instrument in a patient body, comprising the steps of: 
acquiring a two-dimensional X-ray image using an X-iay acquisition system, said two- 
dimensional X-ray image comprising a projection of said medical instrument in 
accordance with a geometry of said X-my acquisition system, 

acquiring a three-dimensional ultrasound data set of said medical instrument using said 
ultrasoimd probe, 

localizing said ultrasound probe in a ref^ntial of said X-ray acquisition system, 
providing a first localization of said medical instrument within a referential of said 3D 
ultrasound data set, 

converting said first localization within said referential of the 3D xiltrasound data set into 
a first X-Ray localiization within said referential of the X-ray acquisition system, 
providing a second localization of said projection of the medical instrument in a 
referential of the two-dimensional X-Ray image, 

mapping said three-dimensional ultrasound data set with said two-dim^tisional X-ray 
image in accordance with a transformation^ which minimizes a distance between a 
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projection of said first X-Ray localization on said two-dimensional X-Ray image in 
accordance with said geometry of the X-Ray acquisilion means and said second 

localization, 

generating and displaying a bimodal representation of said medical instrument in which 
both 2D X-ray image and said mapped 3D ultrasound data are combined. 
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SYSTEM FOR GUIDING AMEDICAL INSTRUMENT IN A PATIENT BODY 



ABSTRACT 

The present invention relates to a medical system comprising a medical instrument to 
be guided in a patient body, ultrasound acquisition means using an ultrasound probe for " 
acquiiing a 3D ultrasound data set and X-ray acquisition means for acquiring a 2D X-Ray 
image, whicli comprises a projection of said medical instrument The system in accordance 
vath Ihe invention fiirther comprises means for localizing said ultrasound probe within a 
referential of said X-Ray acquisition means, means for providing a first localization of said 
medical instrument within a referential of the ultrasound acquisition means, means for 
converting said first ultrasound localization into a first X-Ray localization wititin tiie 
referential of said X-Ray acquisition means, means for providing a second X-Ray 
localization of &e projection of said medical instrument in tiie two-dimensional X-ray image, 
means for mapping said 3D ultrasour»d data set witii said 2D X-ray image in accordance witii 
a transformation, which minimizes a distance between a projection of said first X-Ray 
localization on said 2D X-Ray image and said second X-Ray localization and means for 
generating and displaying a bi-modal r^resentation of said medical instrument, in which said 
2D X-ray image and said mapped 3D ultrasound data are combined. 



Reference: Fig. 1 
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FIG. 11 



